Europaisches 
Patentamt 



European 
Patent Office 



PC T/BE 200 5/0 00 1? J 



Office europeen 
des brevets 



Bescheinigung Certificate Attestation 



Die angehefteten Unterla- 
gen stimmen mit der 
ursprOnglich eingereichten 
Fassung der auf dem nach- 
sten Blatt bezeichneten 
europaischen Patentanmel- 
dung Qberein. 



The attached documents 
are exact copies of the 
European patent application 
described on the following 
page, as originally filed. 



Les documents fixes a 
cette attestation sont 
conformes a la version 
initialement deposee de 
la demande de brevet 
europeen specifiee a la 
page suivante. 



Patentanmeldung Nr. Patent application No. Demande de brevet n° 

04447263.7 



Der President des Europaischen Patentamts; 
Im Auftrag 

For the President of the European Patent Office 

Le President de TOffice europeen des brevets 
p.o. 



R C van Dijk 

BEST AVAILABLE COPY 




Europaisches 
Patentamt 



European 
Patent Office 



Office europeen 
des brevets 



Anmeldung Nr: 
Application no.: 
Demande no: 



04447263.7 



Anmeldetag: 
Date of filing: 
Date de depot: 



26.11.04 



Anmel der/Appl 1cant( s)/Demandeur( s) : 
BARCO N.V. 

President Kennedypark 35 
8500 Kortrijk 
BELGIQUE 



Bezelchnung der Erf 1ndung/T1tle of the 1nvent1on/Tl tre de l 1 Invention: 
(Falls die Bezelchnung der Erflndung nlcht angegeben 1st, slehe Beschrelbung. 
If no title 1s shown please refer to the description. 
S1 aucun tltre n'est 1nd1qu6 se referer a la description.) 

Test or calibration of displayed greyscales 



In Anspruch genommene Prloriat(en) / Priori ty(1es) claimed /Pr1or1t6(s) 
reVendlquee(s) 

Staat/Tag/Aktenze1chen/State/Date/Flle no. /Pay s/Date/Numero de depot: 



Internationale Patentklasslf 1kat1 on/International Patent Classification/ 
Classification Internationale des brevets: 



Am Anmeldetag benannte Vertragstaa ten/Contracting states designated at date of 
flllng/Etats contractants designees lors du depot: 

AT BE BG GH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LU MC 
NL PL PT R0 SE SI SK TR LI 



04447263.7 

EPA/EP0/0EB Form 1014.2 - 01.2000 7001014 



G09G/ 



1 



TEST OR CALIBRATION OF DISPLAYED GREYSCALES 

Field of the invention 

This invention relates to systems for testing displays, to systems for 
determining luminance levels of displays, to systems for calibrating displays, and to 
corresponding methods. 

Description of the Related Art: 

It is known that calibration of a display (in medical imaging also called a soft- 
copy viewing station) is an important component of effective medical imaging 
(including imaging of anatomy, imaging for diagnostic or clinical use, etc.). In many 
cases, there are very small luminance differences between an area of interest (which 
itself may be very small) and the surrounding area. Without proper display system 
calibration, it is possible that the viewing station itself can adversely affect the ability 
to make a proper diagnosis or interpretation of the image being displayed. Particularly 
if using an un-calibrated commercial colour monitor, the low-level shades of grey may 
be particularly hard to distinguish from one another. 

For medical images there have been several guidelines that have been 
developed for calibration. When the American College of Radiology (ACR) and 
National Electrical Manufacturers Association (NEMA) formed a joint committee to 
develop a Standard for Digital Imaging and Communications in Medicine (DICOM), 
they reserved Part 14 for the Grayscale Standard Display Function (GSDF). This 
standard defines a way to take the existing Characteristic Curve of a display system 
(i.e. the relationship between the Luminance Output for each Digital Driving Level 
DDL or pixel value) and modify it to the Grayscale Standard Display Function. At the 
heart of the Grayscale Standard Display Function is the Barten Model. This model 
takes into account the perceptivity of the human eye. Given the black and white levels 
of the display system, it will spread out the luminance at each of the intermediary 
Digital Driving Levels such as to maximize the Just Noticeable Differences (JND) 
between each level. A JND is the luminance difference that a standard human observer 
can just perceive. Calibration has the aim that each DDL will be as distinguishable as 
possible from neighbouring levels, throughout the luminance range, and it will be 
consistent with other display systems that are similarly calibrated. 



A part of DICOM, supplement 28, describes the GSDF (available at 
http://medical.nema.org/dicom/final/sup28^ft.pdf) . . It, is a formula based on human 
perception of luminance and is also published as a table (going up to 4000 cd/m 2 ). It 
also uses linear perceptions and JND. Steps to reach this GSDF on a medical display 
are named 'Characterization', 'Calibration' and afterwards a 'Conformance check*. 
These will be discussed in more detail below. 

Currently the above steps are done in most cases with quantitative methods by 
using a measurement device. In that case the accuracy of the GSDF Conformance 
Check result depends on all kinds of factors like deficiencies of the different devices 
used. This is not important in this context, running a calibration sequence on a stable, 
perfect performing display by using a perfect measurement device, will result in a 
nearly 100% match on the GSDF (there still is a quantisation error present and also 
some instability over time, temperature, ...) . On the other hand, there are known 
solutions to reach the DICOM GSDF without using a measurement device, but by 
using a visual procedure. 

It is also known that The American Association of Physicists in Medicine 
(AAPM) Task Group 18 have guidelines to practicing medical physicists and 
engineers for in-field performance evaluation of electronic display devices intended for 
medical use. They include a family of test images that can be used either visually, or in 
conjunction with test equipment to evaluate the performance of a display system. The 
user needs to tailor the tests based on whether a CRT based or LCD based display 
system is used. (A draft of the existing document can be found at 
http://deckard.mc. duke. eduA-samei/tg 1 8) 

Both CRT-based and LCD-based display monitors have been successfully used 
in medical imaging applications. From a calibration standpoint, a LCD-based display 
is typically more stable than a CRT-based display. A CRT can have variations from 
the electron gun, phosphor, and power supply that will disturb brightness settings and 
calibration. The LCD's primary source of variation is the backlight, although 
temperature, ambient lighting changes, and shock/vibration will also have effects. The 
characteristic curve of an un-calibrated LCD is poor in the sense of DICOM 
conformance, especially in the low-level grey shade regions. It is known to implement 
an initial DICOM correction (typically done via a Look-up Table or LUT), before 
utilizing the display for diagnosis, then make periodic measurements to ensure that the 
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calibration correction is still accurate. Liability concerns mean that institutions need to 
show that they have properly implemented calibration into their medical imaging 
process. This involves the documentation of objective evidence that the viewing 
stations have been properly calibrated. 
5 Known calibration tools include visual test patterns and a handheld luminance 

meter (sometimes referred to as a "puck") or a built-in sensor, to measure the 
conformance to the DICOM standard. These can provide the data to generate a custom 
LUT correction for DICOM Grayscale Display Function compliance. It is known to 
provide calibration software, such as the Image Systems CFS (Calibration Feedback 
10 System) to schedule when a conformance check occurs, and generate a new DICOM 
correction LUT if needed. A log of tests and activity can provide a verifiable record of 
compliance testing, and reduce the need for technicians to take manual measurements. 

It is known from US 529899 to provide an instrumentless display calibration 
system which generates adjustment images with a symbol that appears light on dark 
15 when misadjusted in one direction; dark on light when misadjusted the other way; and 
disappears at the point when the display is adjusted correctly. This can be used to 
calibrate brightness, colour, gamma and sharpness. A plot of brightness versus numeric 
pixel value for a display can be used in various ways. For example, a palette lookup 
can precisely correct for display nonlinearities. In one scenario, a monitor is tested, and 
20 a correction palette generated for the monitor. Since a lot of customers do not have 
calibration equipment or the expertise to re-calibrate a display, monitor aging, repair or 
replacement parts could adversely affect the original calibration. Using the calibration 
system, the brightness curve could be regenerated quickly with the user observing and 
answering a computer prompt, "is the symbol lighter, darker or equal to the 
25 background?". The system can span the entire brightness curve. First, assign light to 
100% white and dark to 0% white. With monitor adjustments fixed, a user can adjust a 
middle tone until the symbol disappears. This gives the numeric pixel value that results 
in a lumens output 50% (middle tone) of the halfway between white and dark. Next, 
the numeric value that provides 75% lumens can be calculated in one of two ways. 
30 First, dark can be assigned to the value just found to give 50% lumens as above, and 
leave light assigned to 100% white. Alternatively, a halftone pattern may be used that 
has 75% white pixels and 25% black pixels. In either case, the null for middle occurs 
at 75% lumens. The curve can be further articulated employing the same technique to 



whatever granularity required. 

As discussed in US published application 20030151781, a known calibration 
proceeds as follows. Firstly, a chart for a correction is displayed on the imaging 
apparatus, for example, a grey chart on which driving levels of the imaging apparatus 
are shown to be different in each step. Then the display of the chart is read by a 
densitometer or luminance meter, and a density or luminance on each step is detected. 
Since the detected density or luminance data (gradation measurement data) correspond 
to the gradations of each step only, a value between each gradation is interpolated 
based on the detected density or luminance data, to make characteristic data of the 
imaging apparatus, using for example, Spline interpolation or Akima interpolation. 

After the characteristic data of the imaging apparatus are obtained, a look-up 
table which shows a relation between the driving levels and the gradation levels of the 
image data is made and the outputted gradation levels are corrected in accordance with 
the look-up table (for example proportional to the driving level to the 2.2 power that is 
an ordinary characteristic of a CRT, as set out in GSDF characteristics of DICOM 
PART 14). 

In case the luminance data after the interpolation have an increasing tendency 
with the increase of the driving level, but the luminance data after the interpolation do 
not show a monotonous increase, a correction step is used. This involves forming the 
second characteristic data of the monitor again, by correcting the detected luminance 
data in a way that the increase of the luminance data after an interpolation becomes a 
monotonous increase. 

There remains a need for improved testing and calibration methods. 

Summary of the Invention 

An object of the invention is to provide improved apparatus or methods for 
testing and calibration of displays, in particular for example medical displays. 

According to a first aspect, the invention provides a system or arrangement for 
testing a display, the arrangement having a test generator arranged or adapated to 
display a series of. test patterns, each at a different luminance or colour, and with a 
predetermined minimum difference of luminance or colour from their background, 
each pattern being unpredictable to a user, and having a test evahiator arranged to 
determine and record if the user has correctly identified each of the patterns. 
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This can enable a more objective test of visibility of luminance or colour 
differences at different luminance levels or colours without needing external 
measuring equipment. The minimum luminance difference can be set to various levels 
according to how stringent the test needs to be. It can be carried out simply by an 
5 unskilled user, and can reduce the risk of the user giving "false positive" responses, 
particularly if a user is in a hurry or is fed up with the testing procedure. 

The arrangement has a means for the user to input information relating to the 
identification of each pattern. For example, as a preferred additional feature of the 
present invention, the test generator is arranged to display alongside each displayed 
10 test pattern a selection of candidate patterns for the user to choose a matching pattern. 

Another such feature is the test generator being arranged to display an array of 
more than one of the test patterns at a time. 

For example the array of test patterns reveals a question arranged such that its 
answer indicates if the patterns have been read correctly. This can enable one user 
15 input to be used for testing multiple patterns, to speed up testing. 

The system" may have means for setting the minimum difference. This can be 
varied to suit different uses. If a test is failed, i.e. some of the test patterns cannot be 
correctly distinguished from the background, the difference of luminance or colour 
between the test patterns and the background can be altered to increase that difference 
20 until the test is passed, i.e. test patterns can be distinguished from the background, and 
the user can make use of the display for less demanding tasks. 

The system may have means for sending a result of the testing, and an identifier 
of the display being tested, to a remote database. 

The system may have means for selecting different levels of background 
25 luminance or colour to test, the selection being spread across the range of the display. 

The system may be arranged to choose a different selection for subsequent 
tests, to test all levels after a given number of tests. 

The system may have a calibrator for altering circuitry used to derive the 
driving levels of the display. 
30 Another aspect of the invention provides a system or arrangement for 

determining an output luminance level displayed by a display for a given luminance 
drive signal, having a detector for detecting a minimal difference of drive signal to 
give a just noticeable output luminance difference at a given high luminance drive 



level, and a processing arrangement arranged to determine an absolute luminance of 
the given high input luminance level from . the . minimal difference and from a 
predetermined human characteristic of visibility threshold of luminance changes at 
different luminance levels. 

This can enable the absolute light output levels to be determined without the 
need for an external or internal sensor, or to check the results of a sensing operation for 
example. This can be useful during conformance checks, or during calibration of the 
display for example. 

An additional feature f this aspect of the present invention is the processing 
arrangement being further arranged to determine a change in output luminance 
corresponding to the minimal difference of drive signal, and use the change in output 
to determine the absolute level, using the human characteristic. 

Another such additional feature is the processing arrangement being arranged 
to use a known transfer function of the display to determine the change in output from 
the detected minimal difference. 

Another such additional feature is the given high input luminance level being 

white. 

Another such additional feature is the detector being arranged to display an 
unpredictable pattern, and receive an indication of whether a user can identify the 
pattern correctly,- then repeat this test with reduced difference, until the user cannot 
correctly identify the pattern. 

Another such additional feature is the detector being arranged to dim the 
display to use a point on the human characteristic with more variation gradient. This 
can improve accuracy, if the display can be dimmed in a predictable, preferably linear, 
fashion. 

Another such additional feature is the system being arranged to determine a 
black output level by detecting a just noticeable difference of luminance at a black 
luminance input level, and deriving the absolute luminance of the black luminance 
input level from the just noticeable difference and from the predetermined human 
characteristic of visibility threshold of luminance changes at different luminance 
levels. 

Another such additional feature is the system being arranged to calibrate the 
display using the determined absolute white and black luminances to adjust a converter 
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of the display used to convert input p values into drive levels. This can ensure small 
differences at the black and white extremes are still visible. 

Other aspects of the invention include methods of carrying out testing or calibrating of 
displays. The method may include the step of marking the display with indicia that the 

5 display is suitable for displaying medical images. 

Any of the additional features can be combined together and combined with 
any of the aspects. Other advantages will be apparent to those skilled in the art, 
especially over other prior art. Numerous variations and modifications can be made 
without departing from the claims of the present invention. Therefore, it should be 

10 clearly understood that the form of the present invention is illustrative only and is not 
intended to limit the scope of the present claims. 

Brief Description of the Drawings 

How the present invention may be put into effect will now be described by way 
15 of example with reference to the appended drawings, in which: 

FIG 1 shows a known display arrangement, 
FIG's 2 to 4 show graphs of characteristics of the display of FIG 1, 
FIG 5 shows a system according to a first embodiment of the present invention, 
20 FIG 6 shows steps according to another embodiment of the present invention, 

FIG 7 shows an example of a test pattern on a background, and candidate patterns, 
FIG's 8 and 9 show steps according to embodiments of the present invention, and 
FIG 10 shows a graph of a transfer function of a display. 

25 Description of the Preferred Embodiments 

The present invention will be described with respect to particular embodiments 
and with reference to certain drawings but the invention is not limited thereto but only 
by the claims. The drawings described are only schematic and are non-limiting. In the 
drawings, the size of some of the elements may be exaggerated and not drawn on scale 
30 for illustrative purposes. 

This invention relates to systems for testing displays, to systems for 
determining luminance levels of displays, to systems for calibrating displays, and to 
corresponding methods. Any type of display may be used, especially those displays 
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which are used for medical imaging. Such displays can be CRT based or fixed format 
or flat panel displays of which LCD, plasma, LCOS, EL displays are only some 
examples. The displays may be non-pixelated such as a CRT display or pixelated like 
the fixed format displays. The displays may also be made up of several sub-displays or 
"tiles" and the complete display of an image is made up of a combination of sub- 
images on the tiles. 

By way of introduction to the embodiments, some terms will be discussed. 
Luminance refers to the amount of light emitted, transmitted, or reflected from a 
surface. The preferred SI unit of luminance is candela per square meter. Other units 
that are still commonly encountered include the foot-lambert (1 fL = 3.426 cd/m 2 ) and 
the nit (1 nit = 1 cd/m 2 ). Luminance is the measurable quantity that corresponds to the 
subjective term "brightness," which is the human perceptual response to the 
appearance of a surface with some luminance. Assuming that other factors (e.g., CRT 
noise, artefacts) remain constant, brightness may be discussed in terms of the just- 
noticeable-difference (JND) index, where 1JND is the luminance difference between a 
target and a uniform background required for the target to be just perceptible. Displays 
with larger ranges between the minimum and maximum luminance values tend to 
provide more JNDs (again, assuming that other display characteristics remain 
unchanged) and therefore improved ability to display targets with small contrast. 

The above-mentioned steps to reach GSDF on a medical display, named 
'Characterization', 'Calibration' and 'Conformance check' will now be discussed, in 
relation to FIGs 1 to 4. FIG 1 shows a display system 10 for producing a standardised 
output, having a converter 20 for converting input p-values (pixel or perceived values) 
into DDL (digital drive levels) so as to apply a correction to correct for nonlinearities 
in the subsequent display 30. These DDLs are fed to the display 30 to produce a 
luminance output. 

CHARACTERIZATION 

In this process the characteristic curve or transfer function of the display 30 is 
determined. This is the natural behaviour of the display 30 (without applying any 
correction). This is normally done by using a luminance measurement device. For all 
applied DDLs (Digital Driving Levels) the output luminance values are captured, this 
results in a transfer function describing the characteristic curve of the display 30. An 
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example is shown in FIG 3. 
CALIBRATION 

Once the characteristic curve of the display 30 is known, the values for the 
5 white, black and ambient luminance should also be known. Now the actual part of the 
DICOM GSDF curve can be determined which is the target curve. Since the current 
function (characteristic curve) of the display 30 and target function are known, a 
correction function can be calculated. An example is shown in FIG 2. This is done in a 
calibration process and the result may for example be stored into a LUT (Look Up 
10 Table). An application which generates a picture to be displayed, draws p-values (pixel 
values), which are translated by the correction function to DDLs and through this 
correction the medical images shown on the display 30 are DICOM GSDF compliant. 
FIG 4 shows an example of the combined effect of the correction function and the 
transfer function of the display 30. 

15 

DICOM GSDF CONFORMANCE CHECK 

For quality assurance, the behaviour of this calibrated display system 10 needs 
to be checked. This is normally done by using (the same or another) measurement 
device measuring all luminance levels for every p-value. The measured curve should 
20 follow the DICOM GSDF curve (set as target during calibration). In practice this 
match is never 100% and so there are different methods to calculate and quantify 
differences between the measured and theoretical target curve. 

EXISTING VISUAL SOLUTIONS FOR CHARACTERIZATION & CALIBRATION 
25 in a prior art solution, the characterization can be done by combining a 

foreground and a background image in combination with a slider to modify an 
intensity of foreground or background. One of the foreground or background image is 
displayed as a checkerboard or line pattern. The goal is to adjust the slider so that the 
intensities of both the foreground and the background image become similar to the 
30 human eye. 

Once this is done for several steps, the characterization for the display is 
known. An assumption on the white, black and ambient light is made and the 
calibration happens by calculating the right correction LUTs. 
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This procedure is very subjective and is dependent of a lot of external factors. 
The biggest problem is that even keeping all environmental and external factors the 
same, running this characterization with 10 different people can give 10 different 
results. 

KNOWN DICOM GSDF CONFORMANCE CHECK 

This check is at present done by using test patterns. When test patterns are 
used, the technical details are normally described in a guide document accompanying 
the software application. These details contain what parts of the test patterns the user 
should look at for detecting display deficiencies (e.g. the lines for geometry 
adjustments can be observed). In a more advanced application the test patterns are 
shown and the user is guided to look for specific details. In that case a question is 
shown on top of the test pattern, the user answers by selecting Yes/No (or he can 
cancel the check). This is a more guided way to do a check. 

There are essentially two groups of known test patterns: 
a) Test patterns with all kinds of combined checks 

For example SMPTE (Society of Motion Picture and Television Engineers) and 
AAPM (American Academy of Pain Medicine) TGI 8 QC test patterns are in this 
category. The DICOM GSDF compliance check is in the best case done by running 
following tests: *r~ 

On SMPTE: The portion of DICOM GSDF compliance in this pattern is only 
about being able to see a 5% box on a black background and a 95% box on a white 
background. A problem with this solution is that 5% and 95% checks are not 
representative for a DICOM GSDF Conformance check. 

For the TQ1 8-QCpattern the-w^rds^QUALITY CONTROL" are displayed for 
3 different levels of grey (dark, medium and light). Every letter of these words is 
showed with a different p-value relative to the background: The 'Q' has a difference of 
14 p-values and the L has only a difference of 1 p-value to the background. The 
description mentions to look for all letters of the 3 occurrences of "QUALITY 
CONTROL". A problem with this solution is that the number of checks are not 
representative for a DICOM GSDF Conformance check: over the total range of 256 p- 
values (assuming an 8 bit system) 3 values are checked with a variance of 1 p-value, 3 
with a variance of 2 p-values, etc. This is not sufficient. A second problem is that it is 
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very hard to see the l L' in a lot of cases, as other test pattern content (like bright white) 
disturbs the adjustment of the eye and can make the 'L' invisible, 
b) Specific test patterns for DICOM GSDF Conformance (e.g. Briggs and Bands with 
text). All current solutions focus on seeing all or some details in one or more test 
5 patterns, but there is no clean solution for obtaining an objective and clear result from 
a visual check. All guided checks post some questions on top of test patterns, but in 
these cases the answers can always lead to non-conformance of the tested displays. 

Conventional methods for visual characterization/ calibration are subjective as 
there is no knowledge about the white and black luminance values. These are needed 
10 for DICOM GSDF calibration. For ambient light a predefined value (like low, 
medium, high) can be accepted. Visual characterization is very subjective: with 10 
persons you can get 10 different results. 

A first embodiment of the invention, illustrated in FIG 5 shows a display 
system 10 comprising a converter 20 and a display 30 as before. In this case, there is 
15 also a test generator 40 arranged to feed test patterns and backgrounds to the display 
30 via the converter 20. A test evaluator 50 receives user input concerning whether the 
test patterns are visible, and checks if the patterns have been identified correctly. The 
test generator 40 and test evaluator 50 can form an example of a detector for 
determining a change in output luminance corresponding to a minimal difference of 
20 drive signal. The test results can be output to a remote database; or can be used for 
calibration. The system can also include parts for calibration. A conformance and 
calibration processing part 60 can be used to adjust the converter 20, typically by 
altering a look up table. Alternatively, the conformance and calibration processing part 
60 can be used to for example adjust parameters in some form of a mathematical 
25 correction function. As will be discussed below, this processing part 60 can use inputs 
from the test evaluator 50, a stored human characteristic 70, and a stored display 
transfer function 80. The processing can be implemented by conventional hardware 
such as a general-purpose microprocessor or ASIC, an FPGA, PLC, PCL, PAL, or a 
desktop computer for example, running software written in a conventional 
30 programming language. 

FIG 6 shows some of the principal steps in a display test according to an 
embodiment of the present invention. The test is started at step 12. At step 22, the 
unpredictable test pattern is displayed over a background with minimal difference in 
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colour or luminance between the test pattern and the background. At step 32, the 
system receives the user identification of the pattern, j.e. an indication of whether the 
user is able to distinguish the test pattern from the background. This is categorized as 
correct, incorrect or do not know, at step 42. The background may be changed at step 
52 and if the test is to continue, at step 62 a decision is made whether or not to repeat 
the test, the method is repeated as from step 22 onwards. 

, After finishing the test, the test results may be sent at step r 72 to a remote 
database, to enable a verifiable test record to be maintained. At step 82, if the test is 
failed, the system may offer a user a less stringent test, using a greater minimal 
difference, or may offer a calibration process. At step 92 is an illustration of using the 
test results for the calibration. An example of this will be described below. At step 94, 
the selection of the backgrounds is cycled so that even if only some background levels 
or colours are tested in one test, after a number of tests, all the levels or colours will 
eventually have been tested. 

According to embodiments of the invention, a sequence of patterns is shown in 
an interactive, objective way: the pattern or sequence is shown, and at every user 
question the user has to make a positive choice of what is determined in the pattern. 
The patterns are unpredictable, and therefore so is the answer. The patterns are 
unambiguous so there is only one good answer possible on every question in the 
sequenqe. A sample of one step in the sequence is shown in FIG 7. The test pattern 75 
is shown barely visible in the middle of the grey background. At the top of the screen, 
a number of candidate patterns 76 is shown, and the user is asked to choose a matching 
pattern or select a "don't know" option. The user input can be by way of a touch 
screen, a pointing device such as e.g. a mouse, or by entering a number or sequence of 
numbers or other character or sequence of characters into a keyboard or in any other 
way. 

The DICOM GSDF Conformance check is done by running through a number 
of steps. In every step a test pattern under the form of a symbol (or image) is shown on 
top of a background. The symbol differs from the background with a number of p- 
values, the number being zero, in which case "don't know" would be the right answer 
to the question (see below), one or more). On top of the image, a popup message is 
shown where the user has to answer the question "which of the patterns is shown?". 
The answer to the question is unique and not ambiguous. The display shows several 
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candidate patterns under the form of symbols (or images) where the user has to pick 
the one he can detect in the shown image. If it is not visible, the user should select an 
option mentioning he cannot determine or see the symbol. 

The number of steps is modifiable. Also the difference of the shown symbol 

5 comparing to the background (in p-values) can be varied. Another variation is that the 
user has to enter what he sees on the screen. The symbol can be a letter or digit (or a 
sequence of letters and/or digits) and the user is asked to type in the letter or digit (or 
the sequence of letters and/or digits) he can see. This again contains multiple steps. A 
second variation is that multiple (or all) steps can be combined in 1 step. An easy 

10 arithmetic can be shown on the image, e.g. "5 + 2 = Every symbol or digit is shown 
on a background with preferably a different p-value level, and with a corresponding p- 
value as foreground, e.g. the "5" is shown with a first foreground level on a first 
background level, and the "2" is shown with a second foreground level on a second 
background level, the first and the second background levels respectively and/or the 

15 first and second foreground levels respectively possibly being different from each 
other. The user is asked to enter the result. The whole screen can be filled with such 
content, so a lot of steps can be combined into one. 

SOLUTIONS FOR CHARACTERIZATION/CALIBRATION 

20 A central service through the internet enables the user to run this visual 

calibration. It is based on the knowledge and test results of multiple users using the 
same or similar type of displays. The concept works with a knowledgebase of display 
characterization, calibration and conformance check information. For every type of 
displays the central storage contains white and black luminance values. This 

25 information is used to do a much more accurate DICOM GSDF calibration. 

Currently more and more home users (radiologists, doctors, referring 
physicians, ...) have remote access to PACS images of the medical facility they work 
for. They now use their default, low-end displays for these purposes. They are not 
allowed to do any diagnostics on medical images on these displays (officially, but in 

30 reality this happens already today). They are allowed to use the displays for reviewing 
only. 

In both cases, the images are not shown correctly. Since it is not a real medical 
display with dedicated features, there is still a need to let the user adjust his low-entry 
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display in a way it can show medical images much better: an easy method to do a 
visual DICOM GSDF calibration and do a. simple visual DICOM GSDF Conformance 
check on a regular time base. 

One of the problems with all existing visual DICOM calibration algorithms is 
that a "guess" is required for the absolute black luminance and the absolute white 
luminance. However, this luminance value (in cd/m 2 ) of the black and white is needed 
to result in a good DICOM calibration. The absolute luminance value is dependent on 
the ambient light (reflection in the display) and the backlight setting and transmittance 
of the display itself. Another problem with existing visual DICOM calibration 
-algorithms is that it is very difficult to determine if the perceptual distance (in p- 
values) between the calibrated greyscale leveis is equal. DICOM indeed requires not 
only that all grey levels are located on a specific absolute luminance curve but also that 
the perceptual distance between these greyscale levels is equal. 

The second aspect of the present invention addresses those problems. FIG 8 
shows an example of steps involved in calibration, which can be carried out by the 
system of FIG 5 for example. At step 81, absolute luminance values are determined for 
black and white input signals. At step 83, the DDL corresponding to a black level is 
used as a starting point for a new LUT. At step 84, the DDL corresponding to a white 
level is used as an endpoint for the LUT. At step 86, the values for greys in between 
are calculated, using the transfer fijnction of the display and the white DDL level as a 
multiplier. An example of a transfer function is shown in FIG 10. The new LUT is 
inserted into the converter 20 and the system can be tested for conformance, as 
discussed above, or using other methods. 

In FIG 9 is shown steps for determining the white and black output luminance 
levels, without having to use an internal or external light sensor. At step 100, a JND in 
p-value at white luminance levels is determined by reducing a difference in luminance 
of a displayed pattern with respect to the luminance of a background, until the pattern 
is invisible (or increasing a difference in luminance of a displayed pattern with respect 
to the luminance of a background, until the pattern becomes visible). This can be 
carried out as described above. Determining absolute value of the white luminance can 
be carried out ignoring the effect of ambient light because the reflected ambient light 
typically is many orders of magnitude smaller in luminance than the light created by 
the display itself when the display is set to (near) maximum DDL. This is done by 
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showing a test pattern with a small difference in video-level from the background. The 
number of greylevel difference (difference in DDL) is then gradually increased until 
the user sees the test pattern. From the (relative) transfer function curve of the display, 
the percentage difference between the levels that are shown can be found. By 

5 decreasing/increasing the difference between the grey levels the percentage difference 
in grey level that can just be observed by the user (step 1 10) can be determined. By 
means of the Barten model the absolute luminance value of the white can now be 
found (step 120) as the point on the Barten model where 1 JND difference equals the 
found percentage luminance difference. The Barten model predicts which differences 

10 in absolute luminance value can be distinguished by the human eye, in other words: 
the contrast sensitivity of the human eye is described in function of absolute luminance 
value. 

Step 130 shows the next stage is to do the same for black. Once the white 
luminance level is known, it is easy to find the absolute black luminance in the same 

15 way (by showing small differences in grey level at dark video levels). The black 
luminance obtained includes reflections from ambient light that cannot be ignored. 

Several models of the human vision system exist. One of them is Barten' s 
model that is used among others in the DICOM GSDF specification. The Barten model 
predicts which differences in absolute luminance value can be distinguished by the 

20 human eye, in other words: the contrast sensitivity of the human eye is described in 
function of absolute luminance value. 

By asking the user if differences between grey levels can be observed it is 
possible to find the absolute black and white level. As a start, small differences in grey 
level are shown at high video levels. In this case the effect of ambient light can be 

25 neglected (with a white luminance of 500 cd/m 2 and a typical reflected ambient light of 
around 1 cd/m 2 the error made is approx. 0.2%). Gradually the difference in grey level 
is decreased until the smallest difference that can be seen by the user is found. Because 
the transfer curve of the display is known (out of a database or stored into the display) 
we know which relative difference (in percent) corresponds to this difference in grey 

30 level. The only thing to do now is to look at the Barten model and find the absolute 
luminance value that corresponds to one JND for that relative difference in luminance. 
In this way we find the absolute luminance of fiuly white. 

For the black level, knowing the transfer curve of the display and also the 
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absolute white level of the display, the absolute difference between the smallest 
difference in grey levels that, can be perceived at step 140 can be calculated. This is 
done in a similar way: by showing small differences in grey level at low video levels 
and increasing/decreasing the difference between the levels of grey until the difference 
can just be perceived. At this moment the absolute white point of the display (in cd/m 2 ) 
is known as is the transfer curve of the display system. It is now easy to translate the 
difference in grey levels (DDL value difference) that can just be perceived into an 
absolute difference in luminance that can just be perceived. For instance: if the 
difference between video level 0 and video level 2 can just be perceived, and if the 
difference between video level 0 and video level 2 corresponds to a 1% variation 
(follows from the known transfer curve), and if the white point of the display was at 
500 cd/m 2 (found earlier in the process), then the absolute difference between video 
level 0 and video level 2 is 5 cd/m 2 . This means that under the present ambient lighting 
the difference of 5 cd/m 2 can just be perceived (corresponds to 1 JND). By again using 
the Barten model the absolute luminance value (display light + reflected ambient light) 
that corresponds to this absolute luminance difference in display light can be found, at 
step 150. Now the absolute luminance of the black level (including reflections of 
ambient light), the absolute luminance of the white level (including reflections of the 
ambient light) and the native panel curve are known. It is to be noted that reflected 
ambient light itself can easily be calculated as absolute luminance of the black level 
(including reflections of ambient light) minus the display light for video level 0 
(known from the transfer curve and absolute white value of the display). It is now 
trivial to calculate the lookup tables required to have DICOM conformance. 

The transfer function of the display is shown in FIG 10 as luminance L for 
different values of DDL. Perceived luminance can be seen as L = a + t(DDL). W, 
where a = ambient light reflected from the display, W = white level, and t(DDL) = a 
native transfer curve of the display. As indicated, there is more uncertainty at lower 
light levels as ambient light plays a larger role. 

In case of lower quality display systems it might be that one difference in grey 
level is already visible. In that case h is not clear whether that difference of one grey 
level is exactly the point where the difference becomes visible or whether for instance 
half of that difference would also be visible. This will obviously result in inaccuracies 
in the absolute luminance estimations for black and white. An obvious solution is to 
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use dithering (spatial or temporal) in case one level difference in grey level 
corresponds to a large difference in percentage luminance value. Of course it is also 
possible to always use dithering in order to make a more accurate estimation of 
absolute luminance values for black and white. It is to be noted that if the transfer 
curve of the display is known, then it is easy to predict what the transfer curve of a 
dithered signal would look like. 

It is to be noted that the DICOM GSDF curve contains some areas where the 
slope of the curve is rather constant over a certain JND range. This is of course an 
unwanted situation for the present model. Indeed: if more absolute luminance values 
exist where one JND corresponds to a measured relative difference in luminance value 
then it becomes impossible to determine the exact absolute luminance value looked 
for. The same thing is valid for determining black level: in that case it could be 
possible that multiple absolute luminance values exist where one JND corresponds to a 
specific absolute difference in luminance value. One solution could be to change the 
brightness of the display in a controlled way. For instance: an LCD typically Has a 
backlight. If the luminance output of the backlight is changed in such a way that the 
resulting luminance values end up at another point of the DICOM GSDF curve where 
the slope of the curve has more variation, then it will be easier to determine the 
absolute black and white point accurately. For instance: suppose that the higher video 
levels of a display system correspond to around 500 cd/m 2 . In that area the slope of the 
DICOM GSDF curve is rather constant. Therefore it would not be clear to which 
absolute luminance value a measured just noticeable relative difference in luminance 
corresponds. However, if the backlight of the LCD is set to l/10 th of its original value, 
then the high video levels of the LCD would correspond to around 50 cd/m 2 . At 50 
cd/m 2 there is a lot of variation in the DICOM GSDF curve (the slope is not constant). 
Because the absolute value of the backlight does not have an impact on the relative 
(known) transfer curve of the display, it is still possible to know which percentage 
difference in luminance corresponds to a difference in a specific number of video 
levels. Therefore it is still possible to find the absolute white luminance value that is 
coming out of the display at that moment. Of course, because the backlight was set to 
1/1 0 th , the absolute luminance value in case of 100% backlight will be 10 times the 
obtained absolute luminance value for white. The same principle holds when 
determining the absolute black value. It is to be noted that multiple technologies allow 
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to accurately set the backlight value to a factor compared to the original value. With 
LEDs (light emitting, diodes) for .instance,, one typically uses PWM (pulse width 
modulation). When the duty cycle of the PWM signal is changed, the light output of 
the LEDs will change linearly accordingly. The same thing holds for PWM driven 
CCFL (cold cathode fluorescent lighting) lamps. By setting the duty cycle of the signal 
to for instance 1/10* of the original value, the output luminance value will also be 
1/1 0 th of the original luminance value. It is to be noted that it is not required to have a 
perfectly linear backlight system. If one knows the behaviour of the backlight system 
in function of its driving signal then it is possible to calculate what the driving signal 
needs to be to have for instance 1/10*, 1/5*, 1/3*, 2, 3, ... times the original luminance 
value. Another solution is to use a (cheap) luminance device. Often a photodiode is 
integrated in the display system (at the front of the display or in the backlight system 
itself). This photodiode is generally not capable of measuring absolute luminance 
values, but allows for performing pretty accurate relative luminance values (because 
the photodiodes are normally very linear or a linearisation correction is applied 
internally). In this way it is possible to set the backlight output luminance to a desired 
factor compared to the original luminance value by making sure that the response of 
the sensor shows the same ratio of new value compared to original value. 

To further improve the accuracy of the algorithm it is possible to add extra 
"measurement points" , for the absolute luminance value. The same procedure -can 
indeed be repeated at non folly-black or folly-white video levels to have extra 
intermediate values of the absolute luminance. The number of measurement points can 
be freely chosen and also the exact DDL value where the measurement is performed 
can be chosen. One criterion to select an appropriate DDL value where an absolute 
luminance measurement is performed could be the variation in steepness of the 
absolute luminance versus JND curve at that DDL level. 

Yet another optimisation is to estimate the number of JNDs in a range of DDL 
values. This reduces the quantization errors. Indeed: suppose at black, folly black is 
shown and the difference in DDL values is determined that corresponds to 1 JND. This 
could be for instance 2 DDL values (so difference between DDL 0 and DDL 2 can just 
be perceived). Now the difference that is needed between DDL value 2 and a higher 
DDL value to correspond again with one JND is also determined. This could be for 
instance 3 DDL values. This means that the difference between DDL value 2 and DDL 
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value 5 can just be perceived. The absolute difference between DDL value 0 and DDL 
value 5 (for instance 4 cd/m 2 based on known transfer curve and white point) can now 
be translated into absolute luminance value by looking on the GSDF curve for an 
absolute luminance value where a difference of 2 JNDs corresponds to an absolute 
5 difference of 4 cd/m 2 . As an additional check it can also be verified whether the two 
differences of each one XND also are valid for that found absolute luminance value. Of 
course the same concept holds for determining the absolute white value or any other 
absolute luminance value. 

Once the calibration is complete the display can be marked with suitable 
10 indicia, e.g. an alphanumeric label that the display is suitable for display of certain 
images, e.g. suitable for display of medical images. 

In summary: a technique to measure the absolute luminance of the display 
white and black by means of visual test patterns has been described. It is to be noted 
that the features described above with reference to FIG 6 to have easy, user-friendly, 
15 online visual DICOM QA check can of course also be used for the calibration steps 
100 and 130. This can provide an easy to use DICOM calibration tool for every 
desktop display that is used for reviewing medical images. 

As has been described, testing a display involves display of a series of test 
patterns, each at a different luminance or colour, and with a predetermined minimum 
20 difference of luminance or colour from their background, each pattern being 
unpredictable to a user, and determining if the user has correctly identified the 
patterns. This can enable a more objective test without needing external measuring 
equipment. Calibrating the display involves determining an output luminance level by 
detecting a minimal difference of drive signal to give a just noticeable output 
25 luminance difference at a given high luminance drive level, and determining an 
absolute luminance of the given high input luminance level from the minimal 
difference and from a predetermined human characteristic of visibility threshold of 
luminance changes. This can avoid the need for an external or internal sensor. This can 
be useful during conformance checks, or during calibration of the display for example. 
30 Other variations will be apparent to those skilled in the art. 



20 



Claims: 

1. A system for testing a display having a test generator arranged to display a series of 
test patterns, each at a different luminance or colour, and with a predetermined 

5 minimum difference of luminance or colour from their background, each pattern being 
unpredictable to a user, and having a test evaluator arranged to determine and record if 
the user has correctly identified each of the patterns. 

2. The system of claim 1, the test generator being arranged to display alongside each 
10 displayed test pattern a selection of candidate patterns for the user to choose a 

matching pattern. 

3. The system of claim 1 or 2, the test generator being arranged to display an array of 
more than one of the test patterns at a time. 

15 

4. The system of any of claims 1 to 3, arranged to set the minimum difference. 

5. The system of any preceding claim, arranged to send a result of the testing, and an 
identifier of the display being tested, to a remote database. 

20 

6. The system of any preceding claim, arranged to select different levels of background 
luminance or colour to test, the selection being spread across the range of the display. 

7. The system of claim 6, arranged to choose a different selection for subsequent tests, 
25 to test all levels after a given number of tests. 

8. The system of any preceding claim, having a calibrator for altering circuitry used to 
derive the driving levels of the display. 

30 9. A system for determining an output luminance level displayed by a display for a 
given luminance drive signal, having a detector for detecting a minimal difference of 
drive signal to give a just noticeable output luminance difference at a given high 
luminance drive level, and a processing arrangement arranged to determine an absolute 
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luminance of the given high input luminance level from the minimal difference and 
from a predetermined human characteristic of . visibility threshold of luminance 
changes at different luminance levels. 

10. The system of claim 9, the processing arrangement being further arranged to 
determine a change in output luminance corresponding to the minimal difference of 
drive signal, and use the change in output to determine the absolute level, using the 
human characteristic. 

11. The system of claim 9 or 10, the processing arrangement being arranged to use a 
known transfer function of the display to determine the change in output from the 
detected minimal difference. 

12. The system of any of claims 9 to 11, the given high input luminance level being 
white. 

13. The system of any of claims 9 to 12, the detector being arranged to display an 
unpredictable pattern, and receive an indication of whether a user can identify the 
pattern correctly, then repeat this test with reduced difference, until the user cannot 
correctly identify the pattern. 

14. The system of any of claims 9 to 13, the detector being arranged to dim the display 
to use a point on the human characteristic with more variation in gradient. 

15. The system of any of claims 9 to 14, arranged to determine a black output level by 
detecting a just noticeable difference of luminance at a black luminance input level, 
and deriving the absolute luminance of the black luminance input level from the just 
noticeable difference and from the predetermined human characteristic of visibility 
threshold of luminance changes at different luminance levels. 

16. The system of any of claims 9 to 15, arranged to calibrate the display using the 
determined absolute white and black luminances to adjust a converter of the display 
used to convert input pixel values into drive levels. 
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17. A display incorporating the system of any preceding claim. 

18. A method of testing a display having the steps of displaying a series of test 
5 patterns, each at a different luminance or colour, and with a predetermined minimum 

difference of luminance or colour from their background, each pattern being 
unpredictable to a user, and determining and recording if the user has correctly 
identified each of the patterns. 

10 19. A method of calibrating a display having the steps of determining an output 
luminance level displayed by a display for a given luminance drive signal, by detecting 
a minimal difference of drive signal to give a just noticeable output luminance 
difference at a given high luminance drive level, and determining an absolute 
luminance of the given high input luminance level from the minimal difference and 

15 from a predetermined human characteristic of visibility threshold of luminance 
changes at different luminance levels, and using the determined absolute output 
luminance to determine a correction for the display. 

20. The method of claim 18 or 19, including the step of marking the display with 
20 indicia that the display is suitable for displaying medical images. 
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ABSTRACT 

TEST OR CALIBRATION OF DISPLAYED GREYSCALES 

5 Testing a display involves display of a series of test patterns, each at a different 

luminance or colour, and with a predetermined minimum difference of luminance or 
colour from their background, each pattern being unpredictable to a user, and 
determining if the user has correctly identified the patterns. This can enable a more 
objective test without needing external measuring equipment. Calibrating the display 

10 involves determining an output luminance level by detecting a minimal difference of 
drive signal to give a just noticeable output luminance difference at a given high 
luminance drive level, and determining an absolute luminance of the given high input 
luminance level from the minimal difference and from a predetermined human 
characteristic of visibility threshold of luminance changes. This can avoid the need for 

15 an external or internal sensor. This can be useful during conformance checks, or during 
calibration of the display for example. 
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